In machine learning theory, problems can be cast in a multi-class regression or classification framework. While the former is the task of decomposing signals into a common basis, the latter is the task of discriminating between different classes. The regression problem reduces to identifying the common sparsity pattern of relevant variables selected from a relatively high-dimensional space. In statistical signal processing, whenever the optimal representation is sufficiently sparse, it can be efficiently computed by convex optimization.
Parametric modeling techniques, such as Gaussian Mixture Models (GMMs) continue to be popular for recognition-type problems in speech recognition. While GMMs allow for fast model training and scoring, training samples are pooled together for parameter estimation, resulting in a loss of information that exists within individual training samples.
Sparse representations (SRs), including methods such as compressive sensing (CS), have become a popular technique for representation and compression of signals. SRs have also been used as a non-parametric classifier for classification tasks. Mathematically speaking, in the SR formulation for classification, a matrix H is constructed including possible examples of the signal, that is H=[h1, h2 . . . , hn].
Compressive sensing that is used for a signal reconstruction is difficult to apply to speech recognition classification problems because a sensitivity matrix that is constructed from training examples does not have to obey restricted isometry properties.